Stabilized Nearest Neighbor Classifier and Its Statistical Properties

Stability has been of a great concern in statistics: similar statistical conclusions should be drawn based on different data sampled from the same population. In this article, we introduce a general measure of classification instability (CIS) to capture the sampling variability of the predictions made by a classification procedure. The minimax rate of CIS is established for general plug-in classifiers. As a concrete example, we consider the stability of the nearest neighbor classifier. In particular, we derive an asymptotically equivalent form for the CIS of a weighted nearest neighbor classifier. This allows us to develop a novel stabilized nearest neighbor classifier which well balances the trade-off between classification accuracy and stability. The resulting classification procedure is shown to possess the minimax optimal rates in both excess risk and CIS. Extensive experiments demonstrate a significant improvement of CIS over existing nearest neighbor classifiers at an ignorable cost of classification accuracy.